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1.  INTRODUCTION 

We  are  developing  a  real-time-multiple-target-tracldng  system  using  a  wide-field-of- 
view  (WFOV)  camera.  The  high  resolution  WFOV  camera  was  conceived  as  part  of  the 
Strategic  Defense  Initiative  Research  at  Lawrence  Livermore  National  Laboratcxy.  The  camera 
system  consists  of  a  lens  made  of  concentric  solid  blocks  of  index  matching  glasses,  CCDs 
arrayed  on  the  focal  plane,  and  a  custcxn  VLSI  image  processor  b)  extract  the  targets. 
References  1  and  2  descri^  the  basic  design  of  the  WFOV  camera  and  die  prototype  system 
that  we  have  constracted.  In  this  paper,  we  will  briefly  review  the  existing  protore  system, 
the  on-going  effort  to  cover  the  full  field  of  view  using  digital  CCD  cameras,  the  production 
of  custom  VLSI  chips  developed  to  extract  centroids  in  r^  time,  and  the  implementation  of 
transputers  to  run  the  tracking  algorithms. 

2.  PROTOTYPE  MULTI  TARGET  TRACKING  SYSTEM 

The  prototype  system  was  constraaed  in  an  attempt  to  study  the  properties  of  the  high 
resolution  ^^OV  lens  and  its  target  tracking  algorithms.  The  f^.8  lens  was  constructed  by 
Perkin  Elmer,  has  an  aperture  of  89  mm,  and  a  ^al  length  of  250  mm.  The  spot  size  over  the 

wavelengths  of  600  ~  670  nm  is  4.3  pm  in  diameter  and  the  measured  transmitted  wavefront 
error  is  <  0.065  waves  across  the  entire  field-of-view  (FOV)  at  632.8  nm. 

Hve  commercial  intensified  CCD  cameras  (Pulnix  840)  on  the  focal  plane  output 
RS 170  video  signals  which  are  fed  into  a  Datacube  image  processor  where  real  time  tracking 
algorithms  are  implemented.  Figure  1  shows  the  schematic  of  the  cameras  and  the  lens 
assembly  within  an  environment  chamber  that  maintains  a  constant  temperature  of  IQPF  ±1°. 
This  assembly  is  mounted  on  a  Contraves  alt-azi  axis  table  widi  a  pointing  accuracy  of  <  2 

prad. 


*  Work  performed  under  the  auspices  of  the  U.S.  Department  of  Energy  by  the  Lawrence 
Livermore  National  Laboratory  under  contract  No.  W-7405-ENG-48. 
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Figure  1:  Wide-Field-of-View  lens  and  five  camera  assembly  for  the  prototype 
tracking  system.  The  system  is  mounted  on  alt-azi  indexing  table;  and  tracks  low  earth 
orbiting  statellites. 
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Figure  2:  Data  Acquisition  for  the  prototype  WFOV  tracking  system. 


The  data  acquisition  system  for  this  prototype  system  is  shown  in  Fig.  2.  The  camera 
output  signals  are  amplified  before  and  after  they  travel  ~100  meter  of  cable  to  reach  the 
indoor  signal  processing  laboratory.  Each  inctnning  image  is  annotated  widi  WWW  timing. 

We  use  Datacube  boards  to  perform  general  image  processing  on  our  images.  Its 
modulariQr  and  open  architecture  m^e  it  easy  to  implement  our  algorithms  to  see  their  real 
time  performance.  The  image  processor  boa^  are  used  to  digitize  the  images,  to  average 
many  frames,  to  create  real  time  histograms,  to  extract  features  and  blobs,  and  to  write  images 
to  disk  in  real  time.  The  Datacube  is  controlled  by  a  SUN  3  via  VME  bus.  We  use  at  least  2 
monitors:  one  for  displaying  the  live  image  signal  directiy  from  the  cameras  and  the  other  for 
di^laying  processed  images  with  overlay  from  the  Datacube. 

We  control  the  Grntraves  table  using  a  GPIB  interface  on  the  SUN.  GPIB  also 
interfaces  a  CAMAC  crate  that  controls  the  camera  on/off  switches  and  the  focusing  motors. 
An  X  window  display  system  is  utilized  to  pass  data  between  SUNs  using  Remote  Procedure 
Calls  (RPC). 

We  view  the  night  sky  mainly  to  track  low  earth  orbiting  satellites.  Details  of  the 
tracking  algorithms  are  explained  in  Reference  2.  In  this  experiinent  stars  are  stationary  over  a 
few  seconds,  and  the  satellites  tqrpear  as  a  star  like  objects  moving  through  the  field  of  view. 
The  goal  of  Ae  tracking  algorithm  is  to  recognize  the  entrance  of  any  moving  targets  into  the 
field  of  view;  to  track  Aem  while  predicting  their  position  and  velocity  using  a  Kalman  Filter; 
and  to  recognize  their  disappearance  fiom  tire  field  of  view. 

hi  (xder  to  subtract  stationary  objects  and  account  for  the  inhomogeneous  sensitivity 
of  the  focal  plane  array,  we  subtract  a  background  image  from  the  incoming  live  image.  The 
background  image  is  obtained  by  averaging  many  fiames. 

The  background  subtracted  image  is  fed  into  a  blob  analyzer  board  which  finds  blobs 
in  real  time.  The  blobs  are  defined  to  be  connected  pixels  above  a  specified  threshold.  The 
threshold  is  determined  by  looking  at  the  mean  and  sigma  of  the  real  time  histograriL  The 
threshold  is  defined  as: 

Threshold  =  mean  +x^  sigma 

where  %  is  set  by  the  user.  The  list  of  blobs  are  compared  with  an  internal  track  list.  If  any 
track  finds  a  blob  within  its  gate,  it  updates  the  tracing  information  using  a  Kalman  Filter 
described  in  Reference  3  and  forms  a  new  gate.  The  blobs  which  do  not  belong  to  any 
existing  tracks  are  listed  as  candidate  tracks.  If  any  candidate  tracks  are  updated  more  than  5 
times  consecutively,  they  are  registered  as  real  tracks.  This  intermediate  track  registration 
method  eliminates  spurious  triggers  from  noise  hits. 

When  a  track  is  not  updated  in  300  msec,  it  is  tagged  as  disappearing  fiom  the  FOV. 
The  program  dumps  the  tracing  information  into  a  file  for  later  analysis  and  frees  the 
memory. 

The  background  image  and  the  threshold  are  constantly  updated  to  take  into  account 
the  back^ound  sky  light  level  changes  caused  by  the  moon  or  clouds  etc.  The  resolution  of 
our  tracing  is  0.5  -  1.0  pixel  and  is  limited  by  Ae  accuracy  of  the  blob  analysis  board  which 
operates  on  binary  images. 

RJLL  FIELD  OF  VIEW  UPGRADE  CAMERA  ASSEMBLY 

We  are  upgrading  the  focal  plane  to  cover  75%  of  the  FOV.  The  new  focal  surface 
assembly  will  be  covered  with  5x10^  pixels  using  23  custom  digital  cameras,  23  custom 
VTSI  chips,  and  23  custom  processing  boards. 


A  schematic  of  camera  assembly  is  shown  in  Fig.  3.  In  order  to  cover  the  large  field  of 
view,  we  use  fiber  optic  reducers  that  demagnify  the  image  by  a  3.8:1  ratio.  The  front  surface 
of  the  fiber  bundle  is  machined  to  match  the  spherical  focal  plane  (curvature  radius  250  mm). 
The  back  end  of  the  fiber  optic  reducer  is  ^ued  to  an  image  intensifier.  The  photo  ou^ut  of 
die  intensifier  goes  through  a  second  reducing  fiber  optic  bundle  glued  onto  a  CQD  array.  We 

use  Thompson  TH  7882  CCDs  which  have  384  x  576  pixels  that  are  23  |xm  x  23  |im 
across. 


Second 
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Figure  3:  Camera  and  reducing  optical  fiber  bundle  assembly.  There  are  23  of  these 
cameras  to  cover  75%  of  the  field  of  view.  The  cameras  ou^ut  digital  pixel  values  at  7  MHz 
rate.  The  images  are  antdyzed  by  23  custom  image  processing  VLSI  chips  and  transputers. 


We  designed  custom  readout  electronics  circuitry  for  these  CCDs.  The  basic  clock  rate 
is  7  MHz  and  the  electronics  will  output  individual  pixel  values  digitally.  The  gain  of  the 
intensifier  can  be  adjusted  by  a  potentiometer  which  is  cwnputo*  controlled  by  CAMAC 
interface.  The  integration  time  of  the  CCD  and  intensifier  is  controlled  by  thumb  wheel 
switches  independently  and  synchronized  for  all  23  cameras. 

4.  AUTOMATIC  CENTROID  EXTRACTOR 

The  output  of  each  camera  is  fed  into  a  custom  automatic  centroid  extractor  (ACE) 
which  finds  the  blobs  in  the  incoming  image  in  real  time.  We  are  developing  a  VLSI  chip  and 
a  printed  circuit  board  that  perform  Ae  connectivity  analysis  of  the  convex  groups  of  pixels 
above  a  specified  threshold 


The  ACE  will  ou^ut  SUMXI  (sum  of  X  position  of  the  blob  weighted  by  gray  value 
of  the  pixel);  SUMYI  (sum  of  Y  position  weight^  by  gray  value);  SUMI  (sum  of  the  gray 
value  of  the  pixels  in  the  blob);  and  SUMN  (number  of  juxels  in  the  blob).  Then,  the  centroid 
is  Xc  =  SUMXI  /  SUMI;  Yc  =  SUMYI  /  SU^  With  the  gray  value  wei^ted  sums,  the 
centroid  accuracy  is  expected  to  be  ^.1  pixels. 

The  ACE  also  has  1  MBit  of  RAM  (Ibit  /  pixel)  to  mask  out  imwanted  pixels.  By 
locating  the  star  positions  and  sizes  accurately,  we  can  mask  out  those  areas  and  exclude  from 
furtho*  analysis.  Although  the  Thomson  CCD  is  only  384  x  576  pixels  we  allocated  1  MBit  in 
order  to  use  the  ACEs  for  any  digital  cameras  up  to  IK  x  IK  size. 

The  prmted  circuit  versitxi  of  the  ACE  electrcmics  has  a  1  KByte  RAM  to  store  the 
resulting  blob  information  which  limits  it  to  5(X)  blobs  /  frame.  However  the  VLSI  chip 
version  utilizes  a  FIFO  memory  so  that  it  can  analyze  an  unlimited  number  of  blobs  in  the 
image  (j^viding  that  the  readout  speed  is  faster  than  the  connectivity  analysis).  The  current 
ACE  ctup  design  consumes  0.5  watts.  More  detailed  design  specifications  for  the  ACE  chip 
and  the  PC  board  are  described  in  Reference  4. 

5.  TRANSPUTER 

The  ACE  blobs  result  is  transferred  to  23  Inmos  T800  transputers  running  the  tracking 
algorithms.  Their  main  functions  are  to  control  the  ACE  registers  and  to  perform  multiple 
target  tracking  algorithms.  We  chose  the  transputers  because  of  tiieir  capability  to  link  to 
neighbraing  transputers.  Figure  4  shows  the  map  of  the  networic  connection  of  transputers  for 
our  23  camera  system.  In  tlus  multi  processor  environment,  when  a  track  is  leaving  the  field 
of  view,  the  transputer  sends  its  target  position,  velocity,  and  gate  to  the  adjoining  transputer 
which  continues  tracking  the  target 

The  individual  transput  interface  board  contains  8  MBytes  of  RAM  memory  to  load 
user  programs.  The  transputers  latch  WWV  time  by  the  frame  valid  signal  from  the  camera. 


Figure  4:  The  map  of  the  network  connection  for  23  camera  system.  Each  rectangle 
represents  one  camera  assembly.  The  arrowed  lines  represent  the  transputer  link  te  the  neighboring 
ones.  When  a  track  is  leaving  the  field  of  view,  the  transputer  sends  its  tracking  information  to  the 
adjoining  transputer  which  continues  tracking  Ae  target 


6.  DATA  DISTRffiUnON 


The  data  distribution  for  this  system  is  diagrammed  in  Fig.  5.  At  the  data  distribution 
station  the  camera  output  is  divided  into  3  paths:  one  to  the  ACE  for  blob  analysis;  one  to  our 
custom  RS170  generamr  to  display  the  image  on  a  monitor,  and  the  other  to  the  Datacube.  The 
Datacube  is  us^  to  perform  gene^  image  processing  on  the  digital  images  including 
determination  of  the  threshold,  control  of  overlays,  and  finding  stars  in  ^e  image.  We  overlay 
a  1  bit  plane  over  the  RS170  display  to  draw  cursors  so  we  can  see  the  performance  of  the 
tracking  in  real  time. 


Figure  5:  The  data  distribution  diagram  for  the  full  field  of  view  upgrade. 


7.  CONCLUSION 

Currently,  we  conq)leted  the  testing  of  prototypes  of  the  camera  electronics,  the  ACE 
board,  and  the  transputer  interface  board.  We  have  now  started  the  production  of  the  23  units. 
We  have  successfully  run  the  tracking  algorithm  across  3  nodes  on  simulated  tracks.  We 
anticipate  that  the  complete  system  be  up  and  running  by  the  summer  of  1990. 

In  conclusion,  we  have  demonstrated  the  capabilities  of  the  WFOV  camera  lens.  We 
have  developai  a  real  time  multiple  target  tracking  algorithm  for  processing  the  signal  from 
this  WFOV  camera  system.  We  are  developing  a  fuU  field  of  view  system  with  >  5x10^  pixel 
elements  using  custom  VLSI  chips  and  a  highly  parallel  image  processing  architecture  which 
we  will  use  to  constract  a  hi^  quality  database  for  low  earth  orbit  satellites. 

We  wish  to  acknowledge  the  contributions  of  the  technical  staff  of  the  WFOV  telescope 
project  including  R.  Allen,  T.  Boyd,  W.  Davis,  J.  Dickie,  F.  Eechman,  J.  Ewins,  D. 
Gustaveson,  T.  Hindley,  T.  Letendre,  B.  Lopossa,  M.  Mercer,  J.  Noonan,  T.  Tassinari,  D. 
Williams. 
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